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Abstract. We introduce bi-parametric fractional integrals of the Erdelyi- 
Kober type that generalize known Garding-Gindikin constructions associated 
to the cone of positive definite matrices. It is proved that the Radon trans- 
form, which maps a zonal function on the Grassmann manifold G n ,m of -Tri- 
dimensional linear subspaces of R n into a function on the similar manifold 
G n fc, l<m<fe<n — 1, is represented as analytic continuation of the cor- 
responding Erdelyi-Kober integral. This result shows that different Grinberg- 
Rubin's formulas for such transforms GR have, in fact, a common structure. 



1. Introduction 

Radon transforms of different kinds have a long history and numerous applica- 
tions; see [Ehj . [GGGj , |GGVj , [HI] , |Ru2] , and references therein. In the present 
paper, we focus on important connection between Radon transforms on Grass- 
mann manifolds and higher rank fractional integrals. Let G n ^ m and G n ^ be a pair 
of Grassmann manifolds of m-dimcnsional and fc-dimensional linear subspaces of 
R™, respectively; l<m<k<n — 1. We use the notation r m and for the 
respective elements of these Grassmannians. The Radon transform of a function / 
on G n>m is a function IZf on G n _k defined by 



(1-1) (ft/)fa)= / f(r m )dr m , 

{r m :T m CT k } 

where € G n fe and dr m is the relevant probability measure. For m = 1, a function 
/ on G nj i can be identified with an even function on the unit sphere S n ~ 1 C K n . 
In this case, TZf represents the totally geodesic transform that assigns to a function 
/ on the unit sphere iS™ -1 its integrals over the set of (k — l)-dimensional totally 
geodesic submanifolds of S n ~ x . Different aspects of the R adon transform (jl.ip were 



investigated by Gel'fand and collaborators |GGR| . [GGSj . Grinberg |Gr| . Grinberg 
and Rubin [GE], Kakehi [Kl], [K2], Petrov [FT], Zhang [ZTiT] . and others. 

There is a remarkable connection between the Radon transform (jl.ip and the 
following Garding-Gindikin fractional integrals associated to the cone Vi of positive 
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definite symmetric £ X I matrices: 

s 

(1.2) (/«/)(*) = -1- [ fWdBtis-rF-V+Wdr, 

o 

(1.3) (/?/)(*) = -1^ / /(r)det(r - S r^ +1 )/ 2 dr. 

M(a) 7 

s 

Here It is the identity £ x £ matrix, 

f h r 

s € Ve; 



iv e n{s-v ( ) Js J(s+v e )n(i t -v e ) 

Tg(a) is the Siegel gamma function (|2.2| : see |Gaj . |Gij . |ORl| for more details. For 
sufficiently good /, the integrals I±f converge absolutely if Re a > (£ — l)/2, and 
extend to all a 6 C as entire functions of a. 

The following result from GR] is of our main concern. A function / on G n<m 
is canonically identified with right 0(m)-invariant function on the Stiefel manifold 
V n>m of n x to real matrices v satisfying v'v — I m , v' being the transpose of v. 
Abusing notation, we write f(r m ) = f(v). Fix an integer £ so that 1 < £ < k—m and 

suppose that / is l-zonal, i.e., f(v) = /o(f), where r — a' e vv'ai, ag = ^ 

It is proved in |GR[ Theorem 4.5], that for m > £, TZf is represented by the 
Garding-Gindikin integral associated to Ve, namely, 

(1.4) (^/)(r fe )= Cl det( S )( f - fe+1 )/ 2 (/; fc - m)/2 /o)W, s = o' i Pi Th v t , 
fo(r) = det(r)C—«-i)/a / o(r) > r 6 V t , Cl = 

Ti(m/2) 

Pr Tfc denotes the orthogonal projection on Tfe. In the case to < £, when rank(r) < £, 
the following formula was obtained in |GR( Theorem 4.5]: 

Im 

(1.5) (Hf){n) =c 2 J det(I m -r) s det{rydr J fo( S 1/2 uru' s^ 2 )du, 

1 = (£-m-l)/2, 6=(k-m-£-l)/2, c 2 = 2-V*"/ 2 - T ^( k / 2 ) 



T m {[k-i)/2) 

Our aim is to show that right hand sides of (|1.4p and (|1.5p can be regarded 
as different forms of the same fractional integral, which is introduced below. The 
latter resembles well known Erdelyi-Kober operators in fractional calculus |SKMj . 

Plan of the paper and main results. Section 2 contains preliminaries. In 
Section 3, we introduce the following weighted versions of the Garding-Gindikin 
integrals: 

(1-6) (J± P fM = ^^ il^Ks), 

gp(r) = det(rf- d f(r), d=(£ + l)/2. 

For to = 1, J± / coincide up to l/T^(/3) with the classical Erdelyi-Kober fractional 
integrals; see |SKM| . The newly introduced normalizing factor l/T^(/3) is needed for 
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analytic continuation of J± f in the /3- variable. We call (|1.6|) fractional integrals 
of the Erdelyi-Kober type. 

If / is good enough, then integrals J± f converge absolutely for Re a, Re (3 > 
d—1, and extend as entire functions of a and (3. We obtain explicit representations 
of J^' m ^ 2 /, m 6 N, provided that Re a > d — 1, see (|3.4p . (|3.5[) . This allows us to 
define J±^ f for i?e a > d — 1 and /3 belonging to the Wallach-like set [FKj 

(1-7) W/ = {o,± l,|,...,^}u{/3: i? e/ 3>^| 

see Definitions 13.41 and 13. 61 

In Section 4, we establish connection between the Radon transform of £-zonal 
functions and integrals (|1.6p . 

Theorem 1.1. Let f be an integrable i-zonal function on V n , m > that is, f(v) = 
fo( r )> r = o~' e vv'o~e. If 1 < £ < k — m, then 

(1-8) (ft/) fa) = I*(k/2) (J^^/ )( S ), 

Formula (|1.8p obviously coincides with (|1.4[) in the case m > £. As we shall 
see below (Remark I4.9p . it also includes (11 .5[) in the case m < I. An analogue of 
Theorem 11.11 holds for the dual Radon transform, see Theorem l4.10l 

Acknowledgement. I am grateful to Professor Boris Rubin for very helpful 
comments and discussions. The research was carried out in the framework of the 
project "Higher-rank phenomena in integral geometry" supported by NSF grant 
DMS-0556157. 

2. Preliminaries 

2.1. Notation. Let Wl n ,m ~ R nm be the space of n x m real matrices x = (xij) 
with the volume element dx = n"=i Ilj=i dxi,j- In the following, x' denotes the 
transpose of x, I m is the identity mxm matrix, and stands for zero entries. Given 
a square matrix a, we denote by \a\ = det(a) the determinant of a, tr(a) stands for 
the trace of a. We use standard notations 0(n) and SO(n) for the orthogonal group 
and the special orthogonal group of M. n with the normalized invariant measure of 
total mass 1. 

Let Se ~ R^(^+ 1 )/ 2 be the space of i x I real symmetric matrices s = (si,j) with 
the volume element ds — Yii<j ^ s i,j- We denote by Vi the cone of positive definite 
matrices in St; Vi is the closure of Vt : that is the set of all positive semi-definite 
I x i matrices. For r 6 Vi (r € 7-^), we write r > (r > 0). Given a,b E Se, 
the inequality a > b means a — b 6 7-^, the symbol J a f(s)ds denotes the integral 
over the set (a + 7-V) n (6 — T 3 ^). The group G = GL(£,M) of real non-singular 
£ x £ matrices g acts transitively on Vi by the rule r — > grg'- The corresponding 
G-invariant measure on 7-^ is (TJ p. 18] 

(2.1) d*r=\r\- d dr, \r\ = det(r), d=(^ + l)/2. 
The Siegel gamma function of 7^ is defined by 

(2.2) r,(a) = / exp(-tr(r))|r| a d.r = n^V 4 ]J T(a - j/2), 
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[Gij , |FK] , [T] . The relevant beta function has the form 

h 

.\f>-d dr _ r e (a)r t (p) 



(2.3) 



\r\ a - d \h-r\ 



T e (a + /3) 



These integrals converge absolutely if and only if Re a, Re j3 > d — 1. For 1 < k < 
£, k e N, the equality (|2~2f yields 

r,(a) T k {a+{k-l)/2) 



(2.4) 



r £ (a + fc/2) 



r fc (a + fc/2) 



For r — (rij) G 7-^, the differential operators acting in the r- variable are defined by 



(2.5) 



(2.6) 



D 4 



det rj. 



d 



dr 



1 if i = j 
1/2 if i ^ J, 



D- = D_ 



(-lYD + , r . 



In the following, all function spaces on a subspace S of Si are identified with the 
corresponding spaces on a subspace of K^ +1 )/ 2 . For instance, S(Sg) denotes the 
Schwartz space of infinitely differentiable rapidly decreasing functions; CD (S) is the 
space of functions / G C°°(S) with supp/ C S. 

2.2. Stiefel manifolds. For n > to, let V n . m — {v G VJl n , m ■ v'v = I m } be the 
Stiefel manifold of orthonormal m- frames in M™. The group 0(n) acts transitively 
on V ntm by the left matrix multiplication. This is also true for SO(n) if n > to. 
We fix an invariant measure dv on V n .m. normalized by 



(2.7) 



dv = 



r m (n/2)' 



|Mu[ p. 70], and denote d*v = o~ n x m dv. The following statement can be found, e.g., 
in [E] . [Mil]. [FK], 



Lemma 2.1. (polar decomposition) Let x € 9Jt nm , n> m. 7/rank(a;) = to, i/ien 
(2.8) x = vr 1/2 , veV ntm , r = x'x£V m , 

and dx = 2- m \r\ { - n - m -V/ 2 drdv. 

Lemma 2.2. (bi-Stiefel decomposition). Let k, ra, and n be positive integers 
satisfying 1 < fc, to < n — 1, k + m < n. Almost all matrices v G V n ,m can be 
represented in the form 



(2.9) 
so that 
(2.10) 



u^-a'a) 1 / 2 
f(v)dv = 



a em, 



k,m ; 



d/i(a) 



dfj,(a 

0<a'a<I m V„_ 



^ G Vn—k,mi 



i{I m - a'afl 2 



du, 



a'afda, 5 = (n - k - m - l)/2. 



For m=l, this is a well known bispherical decomposition |VK[ pp. 12, 22]. In 
the higher rank case, this statement is due to [Herzl p. 495] for fc = to and to 
Grinberg and Rubin |GR| for all k + to < n, see also |Q1R] . |Zhl| . 
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2.3. The Laplace transform. Let z = a + iui, a G Vg, ui 6 Sg, be a complex 
symmetric matrix. Suppose that / is a locally integrable function on Sg satisfying 
f(r) = if r £ Vg, and exp(— tr(aor))f(r) G L}(Si) for some oo G Pi- The integral 

(2.11) (Lf)(z) = J exp(-tv(zr))f(r)dr 

is called the Laplace transform of /. This integral is absolutely convergent in the 
(generalized) half-plane Re z > o~$. Let 

(2.12) {Fg){u) = J exp(tr(iujs))g(s)ds, uj G S e , 

be the Fourier transform of a function g on Si. Then (Lf)(z) = (J-g a )(—uj), 
where g a (r) = exp(— tr(crr))/(r) G ^{Sg) for a > <jq. Thus, all properties of 
the Laplace transform are obtained from the general Fourier transform theory for 
Euclidean spaces [Hcrz], [VD p. 126]. The following uniqueness result for the 
Laplace transform follows from injectivity of the Fourier transform of tempered 
distributions. 

Lemma 2.3. If f\{r) and / 2 (r) satisfy exp(-tr(a r))fj(r) G L l (Vg), j = 1,2, 
for some <7o G Pi, a nd {Lf\){z) — [Lf2){z) whenever Re z > a®, then f\(r) = /2(f) 
almost everywhere on Sg. 

2.4. Garding-Gindikin distributions. Let / belong to the Schwartz space §(Sg). 
The Garding-Gindikin distribution associated to the cone Vg is defined by 

(2.13) g a (f) = jr^r J f(r)\r\ a - d dr, d=(l+ l)/2. 

The integral (|2. 13[) converges absolutely for Re a > d — 1 and admits analytic 
continuation as an entire function of a so that Ga(f) — /(0). The integrals of 
half-integral order have the form 



(2.14) G m /2(f)=*- lm/2 J /(w'w)dw, to =1,2,. 

Tl m , e 

see (EH pp. 132-134], [OR2] . 

2.5. The Garding-Gindikin fractional integrals. Let Q = {r G Vg : < r < 

Ig} be the "unit interval" in Vg. For / G £ X (Q) and Re a > d — 1, the Garding- 
Gindikin integrals are defined by 

s 

(2-15) (/?/)(*) = -L, [ f{r)\s-r\ a - d dr, 



Tg{a) 


(2-16) (/«/)(«) = ^L-jfWr-aF-'dr, 

s 

where s G Q. Both integrals are absolutely convergent. For / G D(Q) and Re a < 
d—1, the analytic continuation of the integrals (|2.15[) and (|2.16[) can be defined by 

(2.17) (I£/)(s) = (I^ +j Dif)(s) if d-l-j<Rea<d-j; 3 = 1,2,..., 



6 



E. OURNYCHEVA 



where D± are differential operators ()2.5|) . (12.61) . 
Lemma 2.4. For f e T>(Q) and a G C, 

(2.18) (I£/)(s) = (7« 9 )(7, - a), g(r) = f{h - r). 

Proof. Since the integrals (I"f)(s) and (J"/)(s) are entire functions of a, it suffices 
to prove (|2.18[) for i?e a > d — 1 . This can be easily done by changing variables 
r —f Ii — r. □ 

Theorem 2.5. If f £ D(Q), then for all m G N, 

(2.19) {I+ ,2 f){ S ) = *~ lml2 J /(s-w'wjdw, 

{we9Tt m ,<:: o)'w<s} 

(2.20) (I m/2 /)(s) = tt^™/ 2 /" /(a + w'wjdw. 
Moreover, 

(2-21) (/£/)(*) =f(s). 

Proof. Formulas (|2. 19|) and (/S./)(s) = /(s) are verified in [OR2 for arbitrarily s G 
7^; (|2.20p and the second equality in (|2.21j) follow from (|2.18[) and the corresponding 
properties of the integral /"/. □ 

Theorem 2.6. If f G i 1 (Q) awd m G N, then (I±^ 2 /)(s) converge absolutely for 
almost all s G Q. 

Proof. It was shown in |OR2j that for nonnegative / and every a£P|, 

a 

' V 2 l f)(s)ds < r( r ^\, \a\ m/2 I f(r)dr. 

T e (m/2 + d) J 
o 

This proves that (I™^ 2 /)(s) is absolutely convergent for almost all s G Q provided 
that / G L 1 (Q). The statement for the right-sided integral is a consequence of 

dUEE}. * □ 

According to Theorems 12.51 and 12.61 the Garding-Gindikin fractional integrals 
can be defined for arbitrary integrable functions / and a belonging to the Wallach 
set 

f 1 3 P — 1 1 f /—I 
(2.22) W«= 0,-, 1, U a: i?ea> 



2' ' 2' '2 2 



3. The generalized Erdelyi-Kober fractional integrals 

Let / be a function defined on the "unit interval" Q={reP<:0<r< Ig\ in 
Vi- For Re a, Re [3 > d — 1, we introduce the generalized Erdelyi-Kober fractional 
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integrals 



(3.2) (J^f)(s) = r^y/zMH^k-r^ 

where s G Q, d = (£ + l)/2. Since 

(3.3) (J^/)(s) = ^^(^OOi ff/j(r) = |?f - d /(r), 

these integrals converge absolutely for almost all s G Q whenever G L 1 (Q). 
Suppose that for fixed /3 satisfying Re (5 > d — 1, the function is infinitely differ- 
entiable and supported in Q. Then the integrals J±'^ f admit analytic continuation 
as entire functions of a so that (J±^ f)(s) = f(s)/Ti((3). 

Our next goal is to show that integrals (|3.1|) . (|3.2|) extend analytically as entire 
functions of /3 and obtain an explicit form of j^ m / 2 f f or m £ N provided that 
Re a > d — 1. 

Theorem 3.1. Let / &e on infinitely differentiable function supported in Q. The 
integrals J± f are entire functions of a and [3. For all m G N, and Re a > d — 1, 

(3.4) (J? m/2 f)( s ) = - — -JiL / | s -cA,r d /(^)^, 

^: o/cj<s} 
\d-a-{3 r 

(3.5) (J"' m/2 /)( S ) = ' ' / Vs + Lo'^fiJ^duj. 



Proof. We apply the Laplace transform technique to prove (13. jj) . One can assume 
that / is a function on Sg which is identically zero outside Q, so that / G §(Si). 
Denote 

(3.6) (JZ<Pf)( S ) = \s\ a+ ?- d (Jl^f)(s). seP e 

It is known (see, e.g., [QR2] ) that for / G T>{P e ), s G Vt, and a G C, 

(3.7) (L^/)(z)=det(z)- Q (L/)(z), i?e z > 0. 

Here, det(z)~ Q = exp(— alogdet(z)), where the branch of logdet(z) is chosen so 
that det(z) = \a\ for real z = a G V m . Hence, for Re (3 > d — 1 and Re z > 0, 

(^/)W = ^^ (L ^ )(2) = ^r7pr / M^xpMr^/MoY. 

Denote F z (r) = exp(-tr(zr))/(r). Then (LJ" ,f} f)(z) = det(z)- Q ^ (3 (F z ), where 
Gp{F z ) is the Garding-Gindikin distribution (|2.13p . which is entire function of (3. 
By (|2.14[) . for m = 1, 2, . . . , we obtain 

(3.8) (Lj"' m/2 f)(z) =ir- lm / 2 det{z)- a J exp(-tr(za/o;))/(u/u;)dw. 



s 



E. OURNYCHEVA 



On the other hand, the Laplace transform of the integral 

-im/2 r 

1(a) = _ , . / \a - Lu'Lu\ a - d f{Lu'Lo)dw 



T e (a) 

can be evaluated as follows 

oo 

-trail r r 

(LI)(z) = -— / f(cu'uj)duj / exp(-tr(zs))|s-w'w| Q - d ds 

M(a) J J 

M m , e 

-im/2 

exp(—tv(zw' w)) f (u' u)dus I exp(— tr (zs))\s\ a ~ d ds 



T e {a) 



By the the well known formula 

exp(-tr(»)H Q ~ d dr = T e (a)det(z)- a , Rea> d-l, 



see, e.g., pferzi p. 479], we obtain 

(LI){z) =ir- em/2 det{ z y a J exp{-ix{zu'u))f(u'u)du. 

Tt m , e 

According to the uniqueness property of the Laplace transform, it follows that 



(J + /)(S) = T^T J \*-<S«>\ a - d fi«>'u)**- 
WeWl m 



This proves (|3.4p . The statement for the right-sided integral is a consequence of 
(|53|) . BUS) , and dot . 

□ 



Theorem 3.2. Lei a function f on Q satisfy the inequality 
(3.9) J \f{Lo'uj)\duj < oo, meN. 

TTiera i/ie integrals (J± m ^ 2 f)(s) converge absolutely for Re a > d — 1 and almost 
all s € Q. 

Proof. It suffices to show that the integrals J^ e ( J±' m ^ 2 f)(s)ds are finite for nonneg- 
ative /, where J± m ^ 2 f are defined by (|3.6[) . For the left-sided integral, changing 
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the order of integration yields 
Jo 



{J a + - m/2 f){ S )ds 



< 



-em/2 



I* (a) 



f{uj ! uj)duj I \s — uj'uj\ cx ds 



n -tm/2 
n- em / 2 T e (d) 



f(tu'uj)duj J \r\ a - d dr 
o 

\h - uj'uj\ a f{uo'uo)duo 



TT- em / 2 Tt(d) 

T e (a + d) 



f{ui'u>)duj < 00, 



as required. Here, we used the substitution r = a 1 ' 2 sa 1/ ' 2 , dr = \a\ d ds |Mu[ pp. 
57-59], to evaluate the integral 



□ 



\r\ a - d dr = \a\ a J \s\ a - d ds = B e {a, d)\a\ a , a = h - u'u>. 
o o 

The proof for j°' m / 2 f j s the same. 

Remark 3.3. For m>£, condition ()3.9() is equivalent to \r\ m / 2 ~ d f(r) G 

Theorem 13.21 allows us to define integrals J±^f for i?ea > c? — 1 and all f3 
belonging to the Wallach set (|1.7j) provided that \r\ f) ~ d f(r) e when Re (3 > 

d — 1, and / satisfies (|3.9p when /3 = m/2, m = 1, 2, ... £ — 1. 

Definition 3.4. XVie left-sided Erdelyi-Kober type fractional integral is defined by 



\d-a-0 



Ti{a)T t {[3) 



f{r)\r\ l3 - d \s-r\ a ' d dr 



if Re (3 >d-l, 



jf-lm/Zlgtd—a-P 



T t (a) 



\s-Lu'uj\ a - d f(u;'(v)dLO if [3 = m/2. 



Remark 3.5. For < m < £, the second integral in Definition 13.41 can be rewritten 
as follows: 



(3.10) (Jt' m/2 f)(s) = c / \q\V- m - X V 2 \I m q\ a - d dq / f {s^uqu' s 1 ' 2 ) du, 



Ve,-, 



c = 2- m TT- lm / 2 /T l {a). 
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Definition 3.6. The right-sided Erdelyi-Kober type fractional integral is defined as 
follows 



(J- f)(s) 



id—a—p 



r t (a)T t (J3) 



f(r)\r 



p-d\ 



I a — d 



dr 



if Re (3 >d-l, 



-lm/2\ s \d—a—[) 



T t {a) 



\s + uj'uj\ a - d f(w'oj)doj if[3 = m/2. 



4. The Radon transform on Grassmannians 

4.1. Definitions. Let l<m<k<n — 1. Given a pair of Grassmann manifolds 
G„. m and G n _k, the Radon transform of a function /(r m ) on G n ^ m is denned by 
(jl . 1|) . The corresponding dual transform of a function ipfa) on G n ,k is 



(4.1) 



<p(T k )dTk, 



{Tfe:T m CT fc } 



Let T m = {xE K" : ('i = 0, ( £ Vn.n-™}, 7* = {a: G R" : fx = 0, £ £ V n ,n-k}- 
Denote by t; = C 1 " the orthogonal complement of £. To give precise meanings to 
integrals JOJ, (|4T|) . we use the following parameterizations: 



r m (u), V S K 



Tfc = Tfe(£), £G K 



The functions f(r m ) on G n!m and (/ ? ( r fc) on G nj fc are identified with the right- 
invariant functions on the Stiefel manifolds V^ j7n and V^n-fcj respectively. Condi- 
tion r m C Tk is equivalent to £'v = 0. The set of all v satisfying the last equation is 

, uj G Vk, m , where g^ is an arbitrary rotation with the 



represented as v = g^ 
property 







(4.2) g^o=t Co = 

This observation leads to the following. 





In-k 



Definition 4.1. The Radon transform of a right-invariant function f(v) on V n 
is defined by the formula 



(4.3) 



mm 



- j f\9i o J d,uj, £ G V n 

v k , m 



-k ■ 



Remark 4.2. Definition (|4.3p of the Radon transform differs from that in |GR) by 
the parametrization of the plane t\. Specifically, in [GRj . (Hf) (r^) = (£""")• 

Let 7„ be an arbitrary rotation with the property 



(4.4) 



J V V = V, V 







eV n 



The set of all £ that obey £'v = has the form £ = -f v 



uGV 7 



n—m,n- 
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Definition 4.3. The dual Radon transform of a right-invariant function ip(£) on 
V n , n -k is defined by 



(4.5) 



(K*<p)(v) 



<p 7. 



u 




The expressions (|4.3|) and (|4.5|l are independent of the choice of rotations : 
£0 — * an d 7t> : — » v, respectively. 



Lemma 4.4. The duality relation 



(4.6) 



f(v)(K*<p)(v)d*v 



is valid provided that either side of this equality is finite for f and tp replaced by \ f\ 
and \ip\, respectively. 



Proof. By (14. 5|) , the left-hand side of (|4.6p equals 



l.h. 



f(v)d*v 



ip 7, 



a 




V„_„ 



^7 y f(0vo)<P ( ^7«o 

SO(n-ro) SO(n) 



7 
J m 



Co d(3, 



where £0 = 
yields 



In-k 





S Vn.n-k- The change of variables /?7„ 



7 

I m 



l.h.s. 



SO(n-m) SO(n) 

<p{PGa)f{pVQ)dp. 



' i 










•MTt 





SO(n) 

Similarly, according to (|4.3p . the right-hand side of (|4.6|) is evaluated as follows 



r.n.s. = 



V n . n -, 



<p(t)d.t J f[gt 

Vk, 



Ul 





<p(PtD)f(Pvo)dp= / p(/3io)f(/3v )d(3, 



where Vq 







SO(n) SO(n) 

€ V n ^ m . This proves the duality relation. 



□ 



Theorem 4.5. TTie Radon transform JZf and the dual Radon transform lZ*tp are 
well defined almost everywhere on V n ,n-k and V n ^ m , respectively, for any integrable 
functions f and if. 
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Proof. It follows from (I46|) that 
(4-7) / (ft/)(Od.£ 

(4.8) 



f{v)d*v, 



v n . n - k v n 

(H*ip)(v)d*v = 

v n , m v n , n - k 

Hence, the integrals (TZf)(^), (7Z*(p)(v) exist for almost all £ and v, respectively. 



□ 

4.2. Radon transform and its dual of invariant functions. The notion of 
£-zonal function on the Stiefel manifold was introduced in |GRj . 



Definition 4.6. Let 1 < £ < n - 1, p E 0(rc - £), g p 



P o 

It 



E 0(n). A 



function f(v) on V nim is called l-zonal if f(g p v) — f(v) for all p E 0(n — £). 



Lemma 4.7. GR, p. 798] Let m + £ < n, m > £. A function f(v) on V n , m is 
£-zonal and 0(m) right-invariant (simultaneously) if and only if there is a function 

fo on Vi such that f(v) a =' fo{r), r = a' e vv'ai = a'fPt v cre., at = 

Here, Pr„cr^ is the orthogonal projection of o~i onto v. 



G V n 



The following theorem establishes connection between the Radon transform of 
a function / of the form f(v) = fo(r), r = o-' e vv'o-£, and the Erdelyi-Kober type 
fractional integrals associated to the cone Vt. Note that for m > I, the function 
fo is defined on Vt, and the integral m )/ 2 - m / 2 j Q exists in the usual sense, cf. 
(|3.ip . For m < £, fo is a function on the boundary dVt, and jl & m )/ 2 >" l / 2 j o j g 



understood in the sense of analytic continuation (13. 4| . 

Theorem 4.8. Let f(v) be an integrable function on V n . m . Suppose that f(v) has 
the form f(v) = fo{r), where r — o' t vv'oi E Vt, and denote s = It — a'^at — 
cr'fPv^ai. If 1 < £ < k — m, then 

(4.9) (nf)(C) = r,(fc/2)(j| fc - m)/2 ' m/2 /o)(s), 

where J^l ™)/ 2 ' m / 2 j s ^ e ^/j e Erdelyi-Kober type operator. 
Proof. By (143)) . 



(4.10) 



Denote 



foiVuy'JdvU;, y u 







Vjfc,. 



«i 

02 



Oi G £Dtfe^, a 2 E Wl n -k,t 
so that y w = a^uj. By Lemma 2.1, a\ — us 1 / 2 , where u E Vk,t and 



h 




13 



Hence. 



v fe . m v k . m 



where uq = 



h 




£ Vfc,£. Applying the bi-Stiefel decomposition 



v^-b'b) 1 / 2 
according to (|2.10p . we obtain 



&k ,■ 



6 e SDL> ,7)1. 1 



|I m - 6'6| (fc - m ^ 1)/2 / (s 1/2 6&' S 1/2 )d6 



{fce5H,,, m : 0<6'b</ m } 



Pk-e,m 



{ a e9Jl m , f : o<j/'a</a 
The change of variables z = ys 1 / 2 , <iz = s| m / 2 ch/, yields 

-m/2 



(w/)(0 



0~k-t,m \S\ 



0~k-l,m \S\ 



VV» e -l/2|(fc-m-£-l)/2 f „ / / 



{ze9Jt m ,<!: 0<z'z<s} 



f-fc+l)/2 



{ze9JT ITM : 0<z'z<s} 



Owing to p.4p . this gives 



(ft/)(£)=c(J + 2 ' 2 / )(s), 



where 

By (EH), 
(4.11) 

which yields c = I^(fc/2). 



^ m / 2 a fe _,, m r,(^) _ r £ (^™)r m (§) 



CTfe - 



r f— 1 



r m (|) r ^(|) 



□ 



Remark 4.9. It follows from (13710]) and (flTT]) that for to = 1,2,... I- 1, formula 
P~g]) coincides with ([Oi l. 

Let us consider the dual Radon transform. 

Theorem 4.10. Let ip(^) be an integrable function on V„,n-fc- Suppose that 
has the form </?(£) = fo(s), where s — o~' e t;£,'o~i G Vi, and denote r = Ii — a^vv'ai — 
a'fPT v ±a£ . If 1 < I < min{/c — m, n — to}, £/ien 



(4.12) 
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Proof. By (@~5 



(4.13) {TZ*if)(v)= / ip Q (z u z' u ) d r u, z u = a'ff v 



Denote 



«i 

«2 



a 2 e 971 



so that z u = a^u. By Lemma 2.1, a\ = lor 1 / 2 , w € V n - m ^ and 



o o 



7„ = (T^Pr^iCf. 



Therefore, 

(n*<p)(v) = 



V n - r 



V n - r 



V n - r 



ipoi^uu'ai) d*u 
(po(r 1 / 2 w'uu'wr 1 / 2 )d*u 
(po(r 1 / 2 WQUu'w r 1 / 2 )d*u, Wq — 



h 




Applying the bi-Stiefel decomposition 



miln-k-b'b) 1 / 2 



b<EWle, n -k, niey, 



n — rn—£,n — k j 



we obtain 



where c = cr„_„ l _^ n _ fe / cr «-m,™-fc- The change of variables z = yr 1 / 2 , dz = \r\^ k ^/ 2 dy, 
gives 



(Tl*<p)(v) = c\r\<- k - n ^ 2 



\h 



-r-^z'zr- 1 / 2 ^- 



- 1 ^ 2 <p (z'z)dz 



c | r |(m-n+£+l)/2 



{z£Wl n _ kt e: 0<z'z<r} 

J \r-z'z\ {k - m - t -^' 2 ^{z l z)dz. 

{ze9Jl„- k x- 0<z'z<r\ 

According to (|3.4|) , we obtain 

(ftV)(«)=ci (J^^^ )(r), 
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where 

—ilk— n)/2„ T-i / k—m\ -p f k—rn \ji I n—m \ 



C\ = 

&n—m,n- 



Vn-k\ 2 > 



By 

■p / n—m—t \ -p / k — m \ 
i »-fc( 2 j = 1 A~ 2~J 

in-fcl — j Ml, - 5— J 

and therefore, c = 1^ ((n — m)/2)). 

□ 
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